From Finite-System Entropy to Entropy Rate for a Hidden Markov Process
نویسندگان
چکیده
منابع مشابه
Taylor Expansion for the Entropy Rate of Hidden Markov Chains
We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...
متن کاملRelative Entropy Rate between a Markov Chain and Its Corresponding Hidden Markov Chain
In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...
متن کاملThe Entropy Rate of the Hidden Markov Process
A stochastic process which is a noisy observation of a Markov process through a memoryless channel is called a hidden Markov process (HMP). Finding the entropy rate of HMP is motivated by applications in both stochastic signal processing and information theory. The first expression for the entropy rate of HMP is found in 1957 [1]. This expression is defined through a measure described by an int...
متن کاملHidden Markov Process: A New Representation, Entropy Rate and Estimation Entropy
We consider a pair of correlated processes fZngn= 1 and fSngn= 1, where the former is observable and the later is hidden. The uncertainty in the estimation of Zn upon its finite past history Z 1 0 is H(ZnjZ 1 0 ), and for estimation of Sn upon this observation is H(SnjZ 1 0 ), which are both sequences of n. The limits of these sequences (and their existence) are of practical interests. The firs...
متن کاملThe Rate of Rényi Entropy for Irreducible Markov Chains
In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Signal Processing Letters
سال: 2006
ISSN: 1070-9908
DOI: 10.1109/lsp.2006.874466